Postgraduate Transfer Report.PDF

This thesis builds upon our current understanding of text reuse by proposing a hypothetical framework of text reuse and applying this abstract definition to a specific domain, that of journalistic reuse. The framework aims to explore a suitable measure of reuse and determine suitable discriminators for document derivation. Although text can be reused verbatim (word-for-word), in most cases, text is reused under certain constraints that cause the text to be rewritten (e.g. time, space, change of tense etc.). Accurately measuring text reuse therefore involves identifying not only verbatim text, but also text that has undergone a number of, potentially complex, rewriting transformations. The proposed reuse framework is hypothetical and applied only to a limited domain. Further application of the framework to wider domains is outside the scope of this thesis. Also involved in measuring reuse are methods to distinguish between text actually reused and that which co-occurs due to co-incidence or chance. For example, in the case of different newspaper versions of the same story, there is a high likelihood that even independent texts will share the same words and phrases. Use of various weighting schemes will help determine whether the effects of chance word co-occurrence can be reduced. At the document-level, other factors are taken into consideration to determine the most likely sources of the reused text, i.e. derivation. The question of what features characterise a derived document versus a non-derived document is addressed and using this information, we hypothetically describe the most likely derivation hierarchy for a set of documents. Classification of documents in the METER corpus is used to test these derivation characteristics. Text reuse is studied in the context of journalistic rewriting and natural language plagiarism but applications include plagiarism detection on the WWW, newswire usage monitoring and control, evaluation of breaches in copyright infringements, finding duplicate or derived documents on the Internet and tracking dissemination of text from academic papers. Using current paraphrasing research and manual analysis of pre-classified texts by a domain expert, we identify and classify a number of common transformations used by journalists in rewriting newswire texts. These are complemented with examples. Measuring Text Reuse Postgraduate Transfer Report Page 1 of 64